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ABSTRACT 

* Research that led to the development of an instrument 

and a proaram for collecting students' evaluations of 
college/university teaching is summarized. The current version of 
■Students' Evaluation of Educational Quality (SEEQ) , which is 
appendedr was developed at the Oniversity of Southern California 
(OSC). (The first SSEQ was developed at the Oniversity of California, 
Los Ancyeles. ) The one-page evaluation instrument contains 
standardized questions, allows for an individual instructor or 
academic unit to design items, and provides for student comments to 
open-ended questions. Research indicates that SEEQ measures nine 
distinct components of teaching effectiveness that have been 
identified in both student ratings and faculty self-evaluations of 
their own teaching. Reliability is good when based upon 10 to 15 or 
more student responses. The ratings have successfully been validated 
aaainst the retrospective ratings of former students, student 
learning as measured by obiective examination, affective corrse 
consequences, and faculty self -evaluations of their own teaching 
effectiveness. Feedback from student ratings, particularly when 
coupled with a candid discussion with an external consultant, 
produced improvement in both subsequent ratings and student learning. 
Data derived from factor analysis, a sample summary report returned 
to individual faculty and department heads, and references are 
included. (SW) 
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Abstract 

The purpose of this study is to describe research that led to the development 
and implementation of SEEQ (£tudents' Evaluations of Educational £uality) . 
SEEQ is an instrument and program for collecting students' evaluations of 
college/university teaching. Research to be described indicates that SEEQ 
measures nine distinct components of teaching effectiveness that have been 
identified in both student ratings and faculty self-evaluations of their 
own teaching. Reliability is good when based upon 10 to 15 or more student 
responses. The ratings have successfully been validated against the 
retrospective ratings of former students / student learning as measured by 
objective examination, affective course consequences, and faculty self , 
evaluations of their own teaching effectiveness. Suspected sources of bias 
to the ratings have been shown to have little impact. Feedback from student 
ratings, particularly when coupled with a candid discussion with an external 
consultant, produced improvement in both subsequent ratings and student 
learning. 
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students' Evaluations of College/University Teaching : 

Research and an Instrument 

Background " 

The purpose of this review is to summarize research that led to the 
development of SEEp (S_tudents' Evaluations of Educational guality) . SEEQ 
is an instrument and a programme for collecting students' evaluations of 
college/university teaching., Research presented in this review is des- 
cribed in greater detail in a series of technical reports and pxJblications. 
Ihis research, in a^idition to guiding SEEQ's development, has also provided 
an academic credibility that is essential in winning faculty support. It 
is hoped that thif< review may serve as both a itiodel and encouragement to 
academic units se;ehing to implement or improve systematic programs of 
students' evaluations. 

Research and development on the first SEEQ, which is substantially 
sixr_ilar to the current version, was conducted at the University of California, 
Los Angeles (UCLA) . -This effort began with a Task Force on the Evaluation 
of Teaching that examined eva.luation practices at UCLA and other universities, 
and made recommendations that included the development of a campus-wide 
program of students' evaluations of teaching. Based upon current practices, 
interviews with students and faculty, and a review of the evaluation literat- 
ure, an extensive item pool was developed. The work done by Hildebrand, 
Wilson and Dienst (1971) at the University of California, Davis was particul- 
arly important in developing this pool of items. Several different pilot 
surveys - each consisting of 50-75 items - vere administered to classes in 
different academic departments. Students, in addition to making ratings, 
were asked to indicate the items they felt were most important in describing 
the quality of teaching. Similarly, faculty were asked to indicate the 
items they felt would provide them with the most useful feedback about their 
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teaching. Students' open-ended comments were reviewed to determine if 
important aspects had been excluded. Factor analysis identified the 
dimensions underlying the student ratings, and the items that best measured 
ec:ch. Reliability coefficients were compiled for each of the evaluation 
items. Finally, after several revisions, four criteria were used to select 
items to be included on the UCLA version of SEilQ- These were: 1) student 
ratings of item importance, 2) faculty ratings of item usefulness, 3) factor 
analysis, and 4) item reliabilities. During the last six years over 
500,000 of these forms have been completed by UCLA students from more than 
50 academic departments in over 20,000 courses. The results of the eval- 
uations are returned to faculty as feedback about their teaching, are used 
in tenure/promotion decisions, and are published for students to use in the 
selection of courses. 

The current version of SEEQ (see Appendix I) was developed at the Univer- 
sity of Southern California (USC) . A preliminary version of the instrument 
was adopted on a trial basis by the Division of Social Sciences, pending the 
outcome of research on the instrument. . On the basis of much of the research 
summarized in this rerview, the current form was unanimously endorsed by the 
Dean and Department Chairpersons in the Division, and its use required in all 
Social science courses. Th-e program was later adopted by other academic 
units at USC, and over 250,000 SEEQ forms have been completed by USC students 
over the last four years. 

Descjaipftion of the Instrument 

The SEEQ survey form is presented in Appendix I. The one-page evaluat- 
ion instrument is self-explanatory, easily administered, and computer scorable. 
The form strives for a compromise between uniformity and flexibility. The 
standardized questions used to evaluate all courses measure separate compon- 
ents of instructional effectiveness that have been identified with factor 
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analysis • , Provision for Supplemental Questions allow the individual 
instructor or academic unit to design items unique to specific needs. 
Adequate provision for student comments to open-ended questions is provided 
on the back of the form. 

A sample of the two-page sxammary report prepared for each course is 
presented in Appendix II (the actual report appears on 8.5 inch x 15 inch 
computer paper). The summary report, along with the completed surveys 
that contain students' open-ended comments, are returned to the instructor. 
Copies of the report are also sent to the Department Chairperson and/or 
the Dean of the particular academic unit. The data upon which the report 
is based is permaniantly stored in a computer archive system by the Office 
of Institutional Studies, the central office that processes the forms. In 
the report, the evaluation factor scores, the overall summary ratings, 
and demographic/ background items are presented on page 1, while the 
separate rating items appear on page 2. Each item is summarized by a 
frequency distribution of student responses, the mean, the standard error, 
and the percentile rank that shows how" the mean rating compares with other 
courses. A graphic representation of the percentile rank is also shown. 
If any supplemental questions were used, a summary of these responses 
appears on a third page. 

The normative comparisons provided in the summary report (the percentile 
ranks) play an important role in the interpretation of the ratings. First, 
students are universally quite generous in their evaluations of teaching. 
The average overall course and instructor ratings are typically about 4.0 
on a one-to-five scale. Second, some items receive higher responses than 
do others - overall instructor ratings are almost always higher than overall 
course ratings. Finally, comparisons are made between instructors teaching 
courses at similar levels (i.e., there are separate norms for graduate level 
courses, undergraduate level courses taught by faculty, and courses taught 
by teaching assistants). Academic units at USC (e.g., the 10 departments 
in the Division of Social Sciences) are given the option of using university- 
wide norros or norms based upon ratings from just their unit. However, 
ratings are only ranked against norms containing at least 209 courses. 
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A longitudinal summary report, summarizing all the available courses 
ever. taught by each instructor is also produced annually. The report 
contains means and percentile ranks for the evaluation factor scores, the 
overall summary ratings, and selected background/demographic items. 
This information is presented separately for each course, and is averaged 
across all graduate level courses and across all undergraduate courses. 
Courses that were evaluated by fewer than 10 students or by less than 50% 
of the enrolled students are not included in the longitudinal averages. 
Important information can be gained from examining this report, beyond the 
convenience of having a summary of all the ratings for each teacher. The 
longitudinal average is not unduly affected by a chcince occurrence in any 
one course offering, and it reflects teaching effectiveness in the range 
of courses that are likely to be taught by a particular instructor. The 
change in ratings over time provides a measure of instructional improve- 
ment. Furthermore, this summary ^ rovides a basis for determining the 
classes in which an individual teacher is most effective. 

In addition to the individual and longitudinal summary reports, other 
studies and special analyses are performed at the request of the Dean and/ 
or Chairpersons. Tliese include requests as diverse as using previ:)us 
ratings for a particular course as a baseline against which to compare 
ratings after an innovative change, a determination of the trend over 
time in ratings of all courses within a given academic department, and the 
use of supplemental questions to query students about their preferences in 
class scheduling. 

Summary of Research 

Factor Analysis 

Factor analysis is used to describe the different components of teach- 
ing effectiveness actually being measured by a set o^^ questions. Its use 
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is pcirticularly important in the development of student evaluation 
instruments, since it provides a safeguard against a "halo effect" - a 
generalization from some subjective feeling about the teacher which 
affects ratings of all the questions. To the extent that all the items 
are contaminated by this halo effect, they will all blend together and 
not be distinguish e*d as the separate components of teaching effectiveness 
that the evaluation form was designed to measure. 

A well developed factor structure is also important to the interpret- 
ation of the student ratings. Broad global ratings averaged across a 
collection of heterogeneous items provide little diagnostic feedback and 
are difficult to interpret. For example. Marsh, Overall and Kesler 
(1979b) showed that while large classes did tend to receive lower ratings 
when averaged across all items, this effect was limited almost entirely 
to the Group Interaction and Individual Rapport factors. Similarly, an 
interview with a student about an earlier version of the evaluation form 
indicated that she had given an instructor lower ratings on several more 
or less randomly selected items because there were no items where she 
could express her sentiment that "the examinations were terrible".. Even 
if particular components of teaching effectiveness seem less important 
to a particular instructor (or academic unit) / their exclusion wHl make 
other ratings more difficult to interpret. 

SEEQ measures nine evaluation factors (see Table 1) . Marsh (Marsh 
and Overall, 1979b; Marsh, In Press) presented a factor analysis of 
student ratings that confirmed the nine factors SEEQ was designed to 
measure, and these findings have been replicated in different academic 
disciplines and in different academic years. Even more convincing support 
came from a study in which faculty in 329 classes were asked to evaluate 
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their own teaching effectiveness with the same SEEQ form that was used by 
their students. Separate factor analyses of the student ratings and the 
instructor self-evaluation both demonstrated the same nine evaluacion 
factors that had previously been identified (see Table 1) - These 
analyses illustrate the replicability of the rating factors and their 
generalizability -.across different methods of evaluation. 



Factor scores derived from the results of factor analytic research 
are an important part of the summaries of the student ratings described 
earlier. Research d: ;cribed in this section is presented in more detail 
in Marsh, Overall & Kesler (1979), Marsh & Overall (1979b), Marsh & Cooper 
(in press) and Marsh (in press) - Further discussion of this issue is 
presented in Marsh {1980b) . 
Reliability 

Reliability refers to the relative lack of random error in student 
ratings, and is a necessary prerequisite for any measurement device.. 
Reliability is assessed by dtitermining the consistency or stability of 
a measure- According to one conceptualization of reliability called 
the interclass correlation^ a reliable item is one in which there is 
agreement among ratings within each class, but consistent differences 
between the ratings of different classes. A similar approach would be, 
to take a random half of th3 students' ratings from each of a large number 
of classes and to correlate their ratings with those of the remaining 
students. The reliability of a given item depends more on the number 
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of students responding than On the actual, jit em. Content . Ilie average 
reliability of SEEQ items is about .90 when based upon 25 students, but 
falls to .74 when based upon only 10 responses and is even lower for fewer 
responses. 

An alternative determination of reliability, called coefficient alpha, 
considers the relative agreement among different items designed to measure 
the same factor. This approach does not include disagreement among 
students within the same class as a source of unreliability, and probably 
results in ai: inflated estimate of reliabilicy. The coefficient alphas 
for the difterent evaluation factors in SEEQ vary betv;een .88 and .97. 

As a consequence of this research, a caution appears on any summary 
report that is based upon fewer than 10 responses. Similarly, these 
courses are not included in the compuatation of the longitudinal averages- 
Data on the reliability of SEEQ items and factors is presented in Marsh 
& Overall (1979b) . 
Long Term Stabilit y 

A common criticism directed at student ratings is that students do 
not have an adequate perspective to recognize the value of instruction at 
the end of a class. According to this argument, students will- only 
recognize the value of teaching after being called upon to apply the course 
materials in further coursework and after graduation. A rather unique 
opportunity to test this notion arose at a California State University 
which had adopted an earlier version of SEEQ. Undergraduate and graduate 
students in the school of management evaluated teaching effectiveness at 
tlie end of each course. However, unlike most programs, the forms were 
actually signed by the students, allowing the identification of individual 
responses. One year after graduation from tlie program (and several years 
after taking a course) the same students were again asked to make 
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"retrospective ratings" of teaching effectiveness in each course, using 
a subset of the original items. Since all evaluations were signed, the 
end-of-term ratings could be matched with the retrospective ratings. Over 
a several year period of time, matched sets of ratings — both end-of-term 
and retrospective — were collected for students in 100 classes. Analysis 
of .th-e two sets of ratings showed remarkable agreement. The average cor- 
relation I'lrelative agreement) between end-of-term and retrospective ratings • 
was .83., Mean differences between the two sets of ratings (absolute agree- 
ment) was small; the median rating was 6.63 for retrospective ratings r.ad 
6.61 for end-of-term ratings. Separate analysis showed tliese results to 
be consistent at both the graduate and undergraduate levels, and across 
different course types. 

Tliis research is described in more detail in Marsh & Overall (1979a) 
and Overall & Marsh (1980b). In related research , Marsh showed ?tlaat 
responses from graduating seniors were similar to the ratings of current 
students. 

Validity — Student Learning 

Student ratings, one measure of teaching effectiveness, are difficult 
to validate since there is no universal criterion of effective teaching. 
Consequently, using cui approach called construct validation, student rat- 
ings have been related to other measures that are assumed to be indicative 
of effective teaching. If two measures that are supposed to measure the 
same thing show agreement, there is evidence tJiat both are valid. Clearly 
this approach requires that many alternative validity criteria be used. 
Within this framework, evidence of the long-- term stcxbility of student 
ratings can be interpreted as a validity measure. However, the most 
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conunonly used criterion has been student learning as measured by perform- 
ance on a standardir,ed examination. 

Methodological probler^s require a ispecial setting for this research. 
Ideally, there are many sections (i.e. different lecture groups that are 
part of the same course) of a large multi-section course in which students 
are randomly assigned or at least enroll without knowledge of who will be 
teacliing the section. Each section of the course should be taught by a 
separate teacher, but the course outline,' textbooks, course objectives, 
and most importantly the final examination, should be developed by a 
course director who does not actually lecture to the students. In tv/o 
separate studies applying this methodology, it was found that the sections 
that evaluate teaching most favourably during the last week of classes 
also perform best on the standardized examination given to all sections 
the following week. Since students did not know who would be teaching 
different sections at the time of registration, and sections did not 
differ on a pretest administered at the start of the term, these findings 
provide good support for the validity of student ratings.;. 

In the second of these studies a set of affective variables was also 
considered as a validity criterion. Since the course was an introduct- 
ion to computer programming, these included such variables as feelings 
of course mastery, plans to apply the skills that were gained from ^he 
course, plans to pursue the subject further, and determination of whether 
or not students had joined the local computer club. In each case, more 
favourable responses to these items were correlated with more favourable 
evaluations of the teacher. 
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These two studies are described in more detail in Marsh, Fleiner & 
lliomas (1975) and Marsh & Overall (1980). Similar findings / using this 
same methodology, are presented in Frey, Leonard, & Beatty (1975) , and 
Centra (1977) , and in studies reviewed by McKeachie (1979) and Marsh 
(1980b, , 

Valid ity — Faculty Self Evaluations ^ 

Validity research such as that described abovo, while supporting the 
use of stndent ratings, has generally been limited to a specialised 
setting (e.g. large multisection courses) or has employed cri.teria (e.g. 
student retrospective ratings) that are unlikely to convince sceptics. 
Thus, faculry will continue to question the usefulness of student ratings 
until validity criteria that are both convincing and applicablp across 
a wide range of courses are utilized. Faculty self-evaluations of their 
own teaching is one criterion that meets both these requirements. Further- 
more, instructors can be asked to evaluate their own teaching along the 
same' dimensions employed in the student rating form, thereby testing the 
specific validity of the different rating factors. In two different 
istudies faculty were asked to' evaluate their own teaching with the same 
evaluation form used by their students, as we,ll as to provide background/ 
demographic information and to express their attitudes toward ttie eval- 
uation of teaching. A letter from the Dean of the Division was also 
sent that encouraged participation and guaranteed confidentiality. 

A majority of tlie faculty (59%) indicated that some measure of 
teaching effectiveness should be given more emphasis in promotional 
decisions. F*aculty clearly agreed that student ratings werr^ useful to 
the faculty themselves as feedback, and a majority of them even agreed 
tliat the ratings should be made publicly available for students to use 
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in course selection. However, they wer.^ more sceptical about the 
accuracy of the student ratings and even more critical of using class- 
room visitation or faculty self evaluations in promotional decisions; 
they were somewhat less critical about colleague examination of course 
outlines, reading lists, and classroom examinations. Faculty also 
indicated a number of potential biases that tliey felt would substant- 
ially affect student ratings. Ihe most frequently mentioned were 
Course Difficulty, Grading Leniency, Instructor Popularity, and Student 
Interest in the Subj»2ct Before Taking the Course. A dilemma clearly 
exists. Faculty are concerned about teaching effectiveness, even to 
the extent of Wcinting it to play a more important role in their own 
promotions. However, many expressed doubts about each of tlie proposed 
measures of teaching effectiveness — including student ratings. 
Before the potential usefulness of the student ratings can be realized, 
faculty and administrators have to be convinced that student ratings 
are valid. 

In the first study, only undergraduate courses taught by faculty 
were considered. Despite faculty reservations about the validity of the 
student ratings, there was considerable student-faculty agreement in tiie 
evaluations of teaching effectiveness. Validity coefficients, tlie 
correlation between student and faculty ratings on the same factors, 
were statistically significant for all evaluation factors (median r = .49). 
Absolute agreement was also assessed by examining the mean differences 
between student and faculty self evaluations. Across all the evaluation 
items the median rating was the same for both students and faculty - 
4.07 ~ and few differences in either direction reached statistical 
significance. 
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In the second study, the same general findings were' replicated with 
a larger sample (329 classes) that included graduate level courses and 
courses taught by teaching assistants (see Table 2) . Student evaluations 
correlated with instructor self evaluations in courses taught by teaching 
assistants (r .= -46), in undergraduate courses taught by faculty (r = .41), 
and even in graduate level courses (r = .39), demonstrating their validity 
at all levels of teaching. Furthermore, a multitrait-multimethod 
analysis (Campbiill & Fiske, 1959) also provided evidence for tlie distinct- 
iveness of each of the rating factors. For example, if a single "general- 
ized rating factor*' underlies both student and instructor ratings, then 
agreement on any particular factor might be a function of this generalized 
agreement and not have anything to do with the specific content of the 
factor being considered. However, if this were the case, the correlations 
between student and instructor ratings on different factors should be 
nearly as high as correlations between ratings on the same factors. 

In 'fact, while correlations between student and instructor ratings 
on the same factors were high (median r = .45) , correlations between 
their ratings on different factors were low (median r =.02). This argues 
for the distinctiveness of the different evaluation factors and for the use 
of multifactor evaluation instri;iments that have been developed with the 
use of factor analytic techniques. The findings of these two studies 
provide further evidence for the validity of the student ratings, suggest 
the possible usefulness of faculty self evaluations, and should be 
particularly helpful in reassuring facrlty about the accuracy of the 
student ratings. • 
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The results of the original study appear in Marsh, Overall & Kesler 
(1979a) , while the findings of the second study are presented in Marsh 
& Overall (1979h) , Marsh & Cooper (in press) ] and Marsh (in press) . 
Relationship with Student , Course & Instructor Characteristics 

It is often feared that variables unrelated to teaching excellence 
may affect student ratings, and the harshest critics ever, suggest that 
faculty can ''buy" favourable ratings by teaching only small courses, 
giving high grades, and requiring little work by students- tVliile these 
attitudes are probably not held by a majority of the faculty, results 
cited earlier suggest that many do feel that student ratings are biased. 
The study of possible biases is complicated' by a number of probloms- 
First is the question of how large a relationship rrust be before it is 
considered practically significant- Second is the problem of how to interpret 
a relationship even if it is substantial- There are generally several 
alternatives and a bias may not be the most likely- For example, the 
positive relationship between student ratings and student learning supports 
the validity of the ratings, and it is unreasonable to say that student 
ratings are biased by student learning- While the question is complex, 
the first step is to determine which* variables are substantially related to 
student ratings - 

The relationship between student evaluations of 511 courses and a set 
of 16 student/course/ins true tor characteristics was examined- 'The set of 
background variables included such things as Class Size, CPA, Teacher Rank, 
Reason fcr Taking the Course, Class Level, Year in School, Expected Grade, 
Workload/Difficulty, and Prior Student Interest in the Si±)ject- Separately, 
each background variable generally explained less than 5% of the variance 
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in any of the student evaluation factors, and there was little indication 
of nonlinearity (see Table 3) . Tlie only variable that consistently 
demonstrated nonlinearity was Class Size — the smallest and largest 
classes tended to be rated most favourably. Several multivariate 
techniques showed tliat 12-14% of the variance in the student ratings 
could be explained by the entire set of background variables. Thi*ee 
background variables were most influential; more favourable ratings were 
correlated with higher Prior Subject Interest, Higher Expected Grades, 
and higher levels of Workload/Difficulty. A path analysis showed that 
Prior Subject Interef?t was most important, and also accounted for one- 
third of the relationship between Expected Grades and ratings. 
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These results show that even the combined effect of the entire set of 
background variables has only a small impact on student ratings, but 
indicated tliat three of these background variables were most influential — 
Workload/Difficulty, Prior Si±)ject Interest, and Expected Grades. Although 
Workload/Difficulty is often suggested as a potential bias, the relation- 
ship foimd in this study was the opposite of the suggested bias. Harder, 
more difficult courses that require more time outside of class receive more 
favourcible ratings. 

Prior Subject Interest, the variable with the largest impact on ratings, 
was examined in greater detail in a separate study'. A similar pattern of 
correlations was fomd between Prior Subject Interest and student ratings 
collected at both UCLA (using the earlier version of SEEQ) and USC. Prior 
Subject Interest was most highly correlated with ratings of Learning/Value 
in both settings, ihe relationship between Prior Subject Interest and Instruct- 
or self-evaluations was also explored in that study. Prior Subject Interest 
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' measured by both student and instructor perceptions — illustrated a similar 
pattern of correlations with both student ratings and instructor self-eval- 
uations. In particular, Prior Si±)ject Interest was most highly correlated 
to both student and instructor ratings of Learning/Value. These findings 
argue that faculty actually are more effective at teaching when working 
with motivated students, and tliat this more effective teaching is accurately 
reflected in the student ratings. 

Hie relationship between student ratings and Expected Grades is subject 
to several alternative interpretations. First, the Expected Grade effect 
was reduced by one-third by controlling for Prior Subject Interest. "The 
best explanation is that Prior Subject Interest caused both better grades 
and a better educational experience- According to this interpretation, 
part of the Expected Grade relationship with student ratings is spurious. 
Second, the Expected Grade relationship can only be considered a bias if 
higher grades reflect "easy grading" on the part of the teacher. If the 
higher grades reflect better student achievement, then the Expected Grade 
relationship may support the validity of the student ratings — better 
ratings are associated with more student learning. At least two facts 
support this interpretation. 'First, Prior Si±)ject Interest is related to 
Expected Grades and it is more reasonable to assume that it affects student 
achievement rather than the instructor's grading standards. Second, 
faculty self evaluations of their own grading standards showed little cor- 
relation with student ratings. In^ reality. Expected Grades probably 
reflect some unknown combination of both "easy grading" and student achieve- 
ment. However, even if Expected Grades do represent a real bias to the 
student ratings, Iheir effect is not substantial, 

Ihese studies show that none of the suspected biases to student ratings 
seem to actually hav^ much impact. Similar findings have been reported 
by Hildebrand, Wilson & Dienst (1971), McKeachie (1979), Marsh (1980b), 
and Remmers (1963). Nevertheless, as a consequence of this research. 
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summary reports describing student evaluations also include mean responses 
and percentile ranks for Prior Subject Interest and Expected Grades 
(see Appendix II) . This research is described in greater detail in 
Marsh (1978, 1980b). Separate studies have examined the relationship between 
student ratings and: 1) Expected Grades (Marsh, Overall & Thomas, 1976), 
2) Class Size (Marsh, Overall & Kesler, 1979b), and 3) Prior Subject 
Interest (Marsh & Cooper, in press) . In related research. Marsh & Overall 
(in press) demonstrated that student ratings are primarily a function of 
the instructor doing the teaching, and not the particular course or the 
level at which it is taught. 

Instructional "Improvement — Feedback from Student Ratings 

There is ample reason to believe that a carefully plcmned program of 
instructional evaluation instituted on a broad basis will lead to the 
improvement of teaching. Teachers, particularly those who are most 
critical of the student ratings, will have to give more serious consider- 
ation to their own teaching in order to consider the merits of an evaluat- 



ion program. llie institution of tlie program and the clear endorsement 
by the administrative hierarchy will give notice that quality of teaching 
is being taken more seriously — an observation that both students and 
faculty will be likely to make. The results of the student ratings — as 
one measure of teaching effectiveness — will provide a basis for 
administrative decisions and thereby increase the liklihood that quality 
teaching will be recognized and rewarded. The social reinforcement of 
getting favourable ratings will provide added incentive for the improvement 
of teaching, even at tlie tenured faculty level.' Finally, the diagnostic 
feedback from the student ratings may provide a basis for instructional 
improvement. As described earlier, faculty at USC indicate that student 
ratings are useful in the improvement of a course and/or the quality of 
their teaching; 80% said that they were potentially useful while 59% said 
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they actually had been useful. However^ the empirical demonstration 
of this suggestion is more difficult to test. 

In two different studies the effect of feedback from midterm 
evaluations on end-of~course criteria was tested. Both these studies 
were conducted with the multi-section course in computer programming 
described earlier. In the first study; students completed an abbrev- 
iated version of the student evaluation instrument at midterm, and the 
results were returned to a random half of the instructors. At the end 
of the term, student ratings of "perceived change in instruction between 
the beginning of the term and the end of the term" were significantly 
higher for the feedback group, as were ratings on two of the seven 
evaluation factors. Ratings on the overall course and instructor summary 
items did not differ, nor did student performance on the standardized 
final examination given to all students. 

Several changes were made in the second study that was based upon 

30 classes. First, midterm evaluations were made on the same evaluation 

1 

form that was used at the end of the course. Second, the researchers 
actually met with the group of randomly selected feedback instnactors to 
discuss the ratings. At this meeting the teachers disc\'.ssed the evaluat- 
ions with each other and with the researchers, but were assured that their 
comments would remain confidential. A third change was the addition of 
affective variables, items that focused on application of the subject 
matter and student plans to pursue the subject. At the end of the term, 
students of the Feedback instructors: 1) rated teaching effectiveness 
more favourably, 2) averaged higher scores on the standardized final 
examination, and 3) experienced more positive affective outcomes than 
students whose instructors received no feedback. Students in tlie 
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Feedback group were similar to the other students in te?nns of botli 
pretest achievement scores completed at the start of the term and the 
midterm evaluations of their teachers. These findings suggest that 
the feedback from student ratings, coupled with a frank discussion of 
f:±ieir implications with an external consultant, can be an effective 
intervention for improving teaching effectiveness. 

The details of these studies have been described in two published 
articles (Marsh, Fleiner & Thomas, 1975; Overall & Marsh, 1980). 



In summary, research described in this study has indicated that: 

1. SBEQ measures nine distinct components of teaching effectiveness 
as demonstrated by factor analysis. Factor analysis of faculty 
evaluations of their own teaching resulted in the same factors. Factor 
scores based upon this research are used to summarize the student ratings 
that are returned to faculty. 

2. Student evaluations are quite reliable when based upon tl\e responses 
of 10 to 15 or more students. Class ratings based upon fewer than 10 
student responses should be interpreted carefully. 

3. The retrospective ratings of former students agree remarkably well 
with' the evaluations tliat they made at the end of a course. 

4. Student evaluations show moderate correlations with student learn- 
ing as measured by a standardized examination and with affective course 
consequences such as application of the subject matter and plans to 
pursue the subject further. 

5. Faculty self evaluations of their own teaching show good agreement 
with student ratings. 



Summary 
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6. Suspected sources of bias to student ratings have little impact. 

7. Feedback from student ratings, particularly when coupled witli a 
candid discussion with an e>^ternal consultant, can lead to improved 
teaching. 



2,2 
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These relationships showed substantial nou-linearity (i.e., quadratic and/or cubic components add at least IZ to the Variance 
Explalaed by the linear relationship and the Total Variance Explained by all components was at least 5Z). 
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12 ;ri::M:' s" 'V.'?! UA\E UAJTUS^! S U:'^W AOU.t^^^tU lAKitit. ^J^ ;lbV^ 1 . 

13 GROUP INTERACTION: l^rUOENTS were EWCOURAGEO to participate in cuss O^SCLlGS;O^JS ::1-.r r.-^.r :r4r r;5:r 
1 4 SIUl"n:t;; :i '/.'LT:1 jMvittO - O S« IAKF TH^JK IOEAS AfiO KfJOVVlEDGE ■ -) -. 5 .3 4: -,5, 
15 i^' UOL-N rs WEBE ENCOUflAGED TO ASK QUESTIONS A WERE GIVEN MEANJNGFUL ANSWERS r:^>:: :<3r; : -4 - 
1 6 r^TuOr-HTS V/tHE ENCOUH-KiE D TC) EXPni;SS Viir.iR OWN IDEAS ANU'OR QUESTION TH[ ir;37K'JCTOH -.l - -.2--' - 3 A' : 5 
17 INDIVIDUAL RAPPORT:' {NSTTJCTOH was friendly towards iNOIVi dual STUDL-NTTS^ ' zQ:: rv^r- r:^: 
1 8 t%..il5,: ft .li MAU: .sUJOj-NTi ' l.fcll VVIfLLV'Mt IN J^rCKINC. MEXP; Af)VK:i: Wi 0\i lU;] I.ILJiL l v.LA::r. \ 2 ■ :3 4 :. 5 

1 9 :^v!^•^■lR,i(:TO^( had a genuine interest in individual students 

20 i:.:-.rh-:-<i-[i^i; \\as AI)^.Oi»ATi.LV ACCf^SUJl.l: TO oTWUf MTS !XJMl^^.■. t:iFrif.t: MCjllfr.; <:)'-. a? M 1 

21 BREADTH: i.^^STRuaon (:o?vTkasted the jmplications of various theories " z-a ::^.r 1:3: .-y^z zj^ 

22 iMMnu'.Tt •^: t :;! •■.(.rjH (I !n! [IoCK(.m(HjIU. t 'I' I f-'.' .iN f'f ;0i AS c; if.'CE »ns urvi.i ■. 1; IN 

23 ir-t".ri;uCTOR PRfJSCNTL-D Pijir .'l ? of VlttVV other THmTJ his/her CvVN when APrnOPRlATll 

24 .ij-^if'i-i^ rr-t- ;>:3touAiM.v r.'ti'.cusiiiLU j!!.jKf^i:NT Df:vELC:fv.!rNry l^l ^ . 1 3 ^ -5 

25 EXAPv/JINATiONS: J^rc^t^MJACK on examinapons/gradeq materialj> v;ar valuaoi.e z:^ - z/^,.: 73. ..4: -^^^r 

26 \': i r<',"i;:> r-} ^VAl UA 1 1 Ni . S] ni ;( ^^I VVOf -^K VVL KE t AIH AND APt-^l iOt'fitAT d 1 5 

27 nXAMiNAiiONS-GRAOED f/lATERiALS TESTED COURSE CONTENT A:- EMPHASIZED BY THE INtiTRUCTOP 

28 ASSIGNMENTS: f-i <iui+-!.n /M.rK^t- ti a?? v-.Li.AhJt;; 

29 ^U^^DING^J. HOMEWORK. ETC. CONTRIBUTED TO APFR£C*ATICN AND UNDEmSTA>)DING OF SUBJECT :il:t rr;./;-. rv^:: 

30 OVERALL: t^O? s :mis ::^^.sr1^^ (:nr:U'Af;r. \vtlH orvirr; c^nur^^r ^ -j ■ 2 ::5 4 5 

31 L^Of:^ THiS iN^vTRUCTOR COMPAflE WiTH OTHEf^ INSTRUCTORS YOU HAVE HAD AT USC? 

STUDENT AND COU.RSE CHARACTERJSTICS (J-JAVE blank if no response_APPLIES) 

32 COiiiit:E DJ^^r-lCsjLT V RELATIVE TO OTHER COURSES. WAS (l-VERY EASY... 3-MEDIUM .. SVERY HARD) 

33 '-'"'i'!-*"'^-- ■.v<?f.>'i oaj:> fi.ra.AiivE ro orHi;;? couH-Sts. was .t-v'tHv lighi .-j mf.oium 'jVEf.v Hi-.vvrj 

34 COURSS TACE WAS ^ i-TOO SLOW . O-ABQUT RIGHT.. S-TOO FAST) , 

35 • t^ v\ ::r"K ^i. o«.'if:£ D t*. n siut I ? CiAS^i oto? ?i to ?■> a 5 ro r *: yTOi:* b lk'\:^^ :.' .1 .c:' .3 ^. .5- 

3g I EVFL OF jf^Tt '^c .^JT IN THE SU'RJCCT PRIOT^ TO ThIiS COURSE fl-VERY LOW... 3.f.fEDIUW . fA'EHY HIGH' 

37 

38 f-V^-'t --Tc D -oPADF. IN THE COuRr,E v'l-r, ?.D, 3-C. 5.A) i:^.- -Q - : c;. z^- 

ao lAK'f ; , :Ht. (v::)uu::r , i ;.*mj(>n Rroi./irn. ,' f.:.von f:LE<;!ivf i . ■.(?;{ tj-'.i ( ^ 

-J •.'>'>u^rO'.t ,.AU f> f fi"i o ?j ( .rrif -lAL iNit ,^^i:,sT * »Mi v;. <;h'irL!nii t mi '/.nit h f.t; j ' - J -1 5 

40 ^EAn iM SCHOOL - FneiMi ■ 2? SOPH. :J) JR. .i: SR.^ GUAD 



i I Av* lii .^NK r>i' >7 Yt.T l-:;t,^ [ft t^HrD AT use 1 ^ 4 o 





- .. 


...^ . 


• - 






3 




1 ■ 

J 


- 

2 


..0. .. 
c> 




..:'J;: 
1 


• "2"" 
" 




• ■ A - 


::l " 






: /V ■ 


1 








-1 
1 






■ 






- -0 




1 


2 






.. 1 .. 


--0-- 




— 1.- 

- - 


■ 1 






^ ■ 


: : 








' ■ '3 " 


■ 2-- 


• 

• 0 


' -A- '■ 


~zV.z 






: '4z:: 


J 


2 ■ 




A 


z:"^: -■ 


-2"-"- 




z-jX" 


: I 


2 




4 


::1 : 


.•:2^r 


- -0 - 




•] 




'i 
.J 






. :.2- ' 


- 3 


-'-» ■ 






•i 


^ • 


- -1 - 

• " < ■- 


zfp:.: 


- -3 - 


-4. - 








*■+• 








A - 


. 1 


2 








rr;?>.-. 


: : 


: 4-: 


■ 1 ■ 




'.3 


.J • 

M 


: ;1 • : 


: 






: 


z 


■ • 




; -J 


2 






■ 


■ 


- ■3 : 




- 1 




. 3 






' ' 


^■3: 




■ 1 ■ 


2 






1 


- XX' 
2 
■2- 


• 

--3 ■ 


A 

4-v 


1 : 


2 


■ ^ 






-/ 








2 


> 


•-r 




2 


3 




;1 


2 




-1 



5 



5 : 

5/ 

58 'J 2 3 4 e5 - 



3e 



INSTRUCTOR'S NAME^j:^:^^';! :^ • ! . ; - DEPARTMENT NAME^: : • !• ; K ' Iv .y ; : : >^CQURSE NUMBER:; 

INSTRUCTIONS 

Tliis evaluation form is intended lo measure your reactions to this instrLictor and course. Results will be reported 
to the Department Chairmen to be used as part of the overall evaluation of the instruclor. These evaluations wit! 
have budgetary and promotional implications so please take it very seriously. When you have finished a desig- 
nated student will pick up the evaluations and take them to the Deparlment Chairperson. Your responses will 
remain anonymous and summaries will not be given to the instructor until after the final grades have been assigned. 

*"''Put Instructor's Name. Department Name and Course Number at top 

(i.e.. Smith, Psychology. 200) " 
'"''^'Use a number 2 pencil, do not use ink. ball point, magic marker, elc 
'^'^^^Biacken only one response for each question and erase any changes completely 



1 7^QPENi-ENDED' COM MENTS 1 
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A>i thw^^l CM/^n.'^eTCPisrics OF rrtts;tN.c.Tp.ijcTOfl/coui<st which you fcel "are JMor^r tMpdRTANT r him/hcr to 'Arm-t ou -.wt^ovm im^vcm .\f^iY ASPtcTS noi CGymio by ratj^g itcms). 
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rUASt IJSl TIU ACIJUlONAt i^P^a TD CLARIFy any Of YOUR fiESPp^^SES OR ?C MAKEiOTHt'fl C0MMC^T5» 



ISSTSOCTCR: DCE, JCHM 
DEPART:-!ES?r: SmE DEPAKKIEOT 
COURSE.' S/\MPLE DEPI 999 



CUSS SCHEDULE NUM3ER: 99999 
m: SPRING 78 



?'.GE 1 OP 2 

MUffiER OF srjoEMis co'-iPLSTiN'G EVM;;vn:»!S: 43 

PERCENTAGE OF ESROLLEO S'lUOEf^fS CO-lFLETINl-G EVALUATIC'JS: 92V 



rt«***********TOIS IS i\ SAMPLE OF CTE Smm REPORT RETURNED TO INDIVIDUAL PACULTi' WJD THEIR DEP^KTMEiJr CHAIRPEfOT************** 



STUDEt^T AND COURSE CHARACTERISTIG : 

FOR ECH QUEOTIOl TOE PERCENTAGE CP SUmmS liMaSG EACH RESPONSE AIID THE MEAN AVERAGE RESPDNSE (IF APPROPRIATE) IS PR£S&-rrEO 
(THESE ST^.TISTICS ARE 3.^SED UEOvN THE KUM3ER OF SHJDEJJIS ACm^vLLY RESPONDIMG TO TliE ITCT) . IN ADDITIO:^, T^E PEXENTAGE OF 
SllCEMTS WHO COiPLETED THE EVALUATION FQS!-1 BUP DID RSSPDND TO A PARTICULAR CUESTIOM IS INDia,TED BY THE "JO RESPDN3B" 
PERCENTAGE, 



PRIOR INI-EREST 
ECT 

1- VERY LO!'? 9% 

2- 9% 

3- ^'tEDIUM m 

4- 121 
SH-ERY HIGtl m 
JJO RESPONSE 0« 
MEAN: 3.39 



OVERALL G.P,A. 

PCT 

1- 3ELCI.1J 2.5 OS 

2- 2.5 - 3.0 27% 

3- 3.0 - 2.4 39% 

4- 3.4 - 3.7 U't 

5- A30VE 3.7 22% 
NO RESPONSE 5% 
imi: 3.28 



EJffBCTED GRADE 
PCT 



0- F 

1- D 

2- C 

3- 3 

4- A 

NO RESPDii'S: 
MEAN: 3.09 



0% 
■ C% 

m 
m 

241 
2% 



REASON IN CLASS 
PCT' 

1^A3 REQRD 444 
2-{W ELECT 37% 
i-GE'!:] ED REQ 2% 

4- MIN/RELTO 2% 

5- GEN I17rR$T 14% 
MO RESPONSE 0% 



YE.AR IN SCHXL 

p:r 

1- FRE5K>1<V; 94 

2- SQF;ia'10RE 35% 
3n]U.NICR 334 

4- SENIOR 234 

5- <3RADtJATE 04 
KO RESPONSE 04 
;^£AN: 2.59 



24 
24 
404 
404 
164 
04 



COURSE WCSKLOAD 
PCT 

1-VEKY LIGHT C4 
2- 

3-MEDIUM 
4- 

5-VERy HEAVY 
NO RESPONSE 
.MEAN: 3.32 



CCURSE 



04 
714 
244 
5% 
24 



1-TOo sl:jv' 

2- 

3-RIGHT 
4- 

5-TOO FAST 
NO RESPONS: 
ym- 3.23 



PACE 


OUTSIDE m/m 


PCT 


PCT 


; 24 


1-0 TO 2 04 


0% 


2-2 TO 5 55? 


714 


3-5 TO 7 324 


194 


4-3 TO 12 134 


' 74 


5-OVER 12 .0% 


;e 24 


m RES?:X\'SE 74 




MEA!}: 2.56 



MAJOR DSPRT!-INT 
PCT 

1- 30C CCI 

2- MAT SCI 

3- !iU''i:rriES 

4- 3USINES5 

5- EDUCATION 

6- ESGINEER 

7- PEP5 ART 

8- PU3 AFFR 

9- CT:-3ER 
0-UNTEC 



724 
24 
9% 
24 

04 
04 
04 
C4 
124 
24 



COOKSE DIFFICUL'^ 
PCT 

1-VERY EASY 
2- 

j-MEOIK'I 
4- 

5-VERY HARD 
NO RESPOfER 
MEAN: ?.64 ' 

mim EVALUATION SCORES: KINt EV7\LUATI0!} FACTOR SCORES, T.CO CVEPALL RATING ITEMS, AND m STUDEOT/CCUR3E CHARACTERISriC ITEI-IS 

TIE NINE EVALUATION FACICR SCORES ARE WEIGHTED AVLRAGS3 0? SEPAR.-.TE RATING ITE:-!S AKD K.AVS A MEfA' AVEP.AGE (ACRCSS AIL U5C 
Ca-RSE*;) 0? 50. for all scores, m STKJS^SD error (SE) is a measure of l^iE RELIABILITY. IT IS S'lALLER {KO.RE F£LIABLE) l\nSJl 
LAICER frj:^2ERS OF STUDEiflS ARE FES?2©ING AIG V;}-£13 THERE IS GPflATER AGREEMEtsT AXCtCG THE STUDS^TS CQ-IPLETING THE EV^iLCA.TIONS. 
. DIFFESEI-5CES OF LESS THAN ^'S SlhimO ERROR ARE TOO SMALL TO BE RruTABLY INTERPRETED. IN GENERAL, EVALUATION'S S/^SED UP^^^ LESS , 
•m^ 10 STUDENTS" RESPONSES OR LESS TliAN 504 OF KE STL'DSNIS ENROLLED IN TOE CLASS SHOULD BE INTERPRETED CAUTIOfJSLY. THE 
PESCEJJriLE RANKS (KHIGi MAY mi EET\-IEEN 0 & 100) AND THE CORRESPONDING GRAffiS SHCK UOH YOUR RATII'JGS COMPARE WITH OTHER COU.qSES 
IN YCUR CCHPARISON GEC'UP. HIGHER PERCENTILE Rf^irKS m MCSE STARS INDICATE HIGHER RATINGS. YOUR CO!-IPARISOM GROUP IS: 
*********** UfDEfGRyxJATE COURSES NOT TAUGHT BY TEACHING S5SIST,V,TS ************ 



EVALUATIOM FACTOR SCORES .'EAN 

:£ARN-IKG VALUABLE LEARNItXJ EXPLRIEfCE, KAS LNTELr.EClUALLY 3TIMULATI^rG/CHALLENGI^)G 53.2 
2?rHU3IA3M raSTR DISPLAYED ENTHUSIASM, ENERGY, -HUMCR & .^3ILITY TO HOLD INTEREST 56.9 
ORGANIZATION CRGANIZATION/CIARITY OF EXPLANATIONS, COURSE MATERIALS, CGJECTIVE3, LECTURES 
;rCOP lNTEHA£T STUDESm EtCOURAGED 'lO DISCUSS, PARTICIPATE, SPJ'iRS IDEAS & .^3K QUESTia^JS 
[NOV. RappCRT INSTRIXTOR ACCESIBI£, FRIL'^DLY, AND INTERESTED IN S'lUPE^vTS 

PRESENTATION OF BRQAD BACKGRD, CO>CEPrS & ALTERKTiTIVS A.opRCAC!]ES/THECRIES 
STUDENT FL'iCEPriO>o OF yMljZ L FAIRNESS OF FaAMS/GRADED f-lATERIAt^ 



SE 
+/- 
1.745 
i.506 
61.3 1.726 
51.3 2.160 
48.7 2.362 



3READTH 

2(a:-ii:vation3 
\ssignments 

vORKLOAD/DIFF 



VALUE OF ASSIQMENTS IN ADDIfCG APPR£CIATIC1^!/U:•EERSTAi^DING TO COURSE 



■55. 
50. 
59. 



1 2.274 
9 2.188 
9 1.639 



RELATIVE COURSE IvCRKLOAD, DIFFICULTY, V^CZ, 
OVERALL SU>!M.ARY ITS-IS 



«JD OUTSIDE HOURS RETIRED 



51.5 1.515 



^riL 

RA!« 
80 
74 
90 
53 
43 
70 
55 
S8 
50 



GRAPH OF 4TILE RANTC 

0 12 3 4 5 6 7 8 9 

***************** e 

*************** ! 

******************* * 

*********** "> 
********* 

*************** d 

************ s 

****************** s 

*********w**x 0 



OVERALL COURSE HQf DOES THIS COURSE COMP.^^RE '.(TTH aHiERS AT U.S.C.? (C^JESTION 30) 4.44 0,102 

JVEPALL IN3TR HCH DOES THLS' INSTRUCTOR COMP<\RS WITH OTHERS AT U.3.C.? (QUESTION 31) 4.61 0.089 



83 



***************** c 
***************** e 



IMPOHTAIsT STUDENT/CCURSE CK^.=!ACTESISTICS . - 

£VEL OF ISTEP.EST IN SUSJECT PRIOR TO THIS CCURSS (1-VLRY LO^? ... 5-VERY HIGH) (QUESTION 35) 3.39 0.191 49 
3(?ECTED GRADE IN TliS CCURSS (0-P, 1-D, 2-C, 3-3, 4-A) QUESTION 38) ' 3.09 0.428 30 

SREPAJIED BY DR. KER3ERT W. ^LaRSH, OFFICE OF INSnTOTia-LfiL SlUDIES, USC; L.A., CA 90C07 TELEKONS: (213)741-6503. 



********** 
******* 



IKSTRUCTOR: DOE, JOHN ClfSS SCHEDULE M^DBR: 99999 PftCE 2 OF 2 

DEP^R■I:•lE^^^.• sample DCPART^&T TER^: spring 78 number of STIIDENTS CWPLETIN'G EWXiJ'VnCN'S: 43 

CCuRSE; S^tPLE DE?T S99 PERCEtTTAGE OF EroOLLED S'^JDENTS CmpLETIf;G EV;^ATI0:!3: 921 

EVALOATICM IT£>;S (SaME QL'£STI03 RWE SEEN ASfJKE^IATED) : 

FOR EACH C^^ESTIOS, THE PERCE>r,\GS OF STUDEtrre M^KT^;<; EACH RF5P0NSE, IKE ME^^' .'.VEaAGS RSS?0>'^E, .\N"D THE STANWnD EPO (SE) OF 

T-z res?->:.:-;es are PHESEicrED (thf^'^e statistics afo based upon tiie acii'^l n^TiIier is 

ADRITIG^:, "niE PERCENTAGE OF STUDENTS WiD CCMPL'-n::) Till: EVALQATICM FCR'! BLT DTD :iOT RE5?;r,t' TO A F^VRTICULAK C\JSSTI0N IS 
INOIGTra Irt THE "i€ KSSP" CCLCTN. DIFFSFi';J:E: IN MFAV AVERAGES Tr'AT ARE LESS T":!''!: 0\'3 STAN'a\RD Ei'JffiR (SEE P/^GE Q'lE 
F0« A rE3CRI?riO>i) /X-<:E S^!A.LL TO SC RELIABLY INT'EI^PFNErSD. 2^ G£NE!^\L, EVriLUATIOMS BASED UPOM * LESS '.H\S' 10 STUDENTS'* 
RESPC^ISES, EVAXUATICS3 3,^SED UPON LESS Vm 5fl% OF TliE CL»SS, ANO EVALUATION ITSIS WillCH ■.•;ERE FF^CUEiTTLV LEFT BWi'jK SHCt'L^ BE 
INTEW'PETED I^XUnOuGLi'. TiiE ?"i;CE^rrILE R'^NKS ("vHICK V'uRY nCWEEN 0 S ICU) •V,'!) THE GKAPi'iS Sii?.',' irj.-: KUK EVALUATIONS COt-lP.^RE 
WITH OZim 'CXTj'ES IN VOLT. CWF.^HTSO*: G?.C:J? ;'i:G1i;R Pb?OE^:^LE PAN'KS A'^H >;:■•«£ STAR''. lN',70Tii 
rccR CC-'iP-V^ISOt^ GRCL'P IS: ***-♦'«***«* L'NDEnGPAa'ATU C0L9SES NOP TA'JG:!T BY TEACHING AS:^ ISTANT-J 

PEi;C!l-W%GE KE3roMDUr3 R-VNK RELr\TIVr TO YOL"il 

cainvRTSOi! grcu? (see above) 
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14. Lc^.iVMv*'^ r.'Viiii i/^aNo 
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20- A:Ci:^i'3r3If: CLTvIiG office !:RSAV"T^R cl^^s 


c 






50 


34 


12 


4.17- cm 


63 


IFTtlit*********** 
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21. i:<5TR COrrWoXF.O ikplicatio^k c? variocs thecries 


0 






36 


43 


2 


4.13 C.126 


57 


**:(^*******<r*** 




22. ICTR ?R2SENT!-:n BACIGROUMD 0? IDFl-^S/raCSPTS 


0 


c 




40 


51 


0 


4,'!1 O.ICO 


84 


, ***********.tt***** 




23. PRHSENTID POINTS Or \7E;; OlfiCR TIL^N G-^? 


c 


2 




45 


33 


2 


4.09 0.121 


53 


*********** 




24. INdTR discussed CURRENT DEVEWPI'lSTi'S IN FIELD 


0 


2 


15. 


35 


47 


0 


4.25 0.125 
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************ 




E:<P>HiV^TIC:c? 






















25. FEEDB/^.CK CM E>;vl5/G?ADED y;.\TE?JAL5 V^/^S 'vAIL'^3LE 


2 


12 


31 


40 


14 




3.51 0.14S 


38 


A******* 




26. ^£:^::^D cf Ey;i[;;^io^i >:a3 fahj s A??f????iATE 


(; 


5 


21 


52 


21 
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3. 05 G.12i 


56 


************ 




27. cwm:ed :•^^•rf:^^AL3 tested c^^5;E :auE::r a:; E';?;iA3:zED 




C 


2^ 


43 


33 




4.09 0.115 
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ASSIGS'iEl.lS 
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23. P.ECUIRED KE;'J)U?:;S/?i:X'rS WERE VAI;.V\3LE 
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0 


7 


49 


44 


0 


4.35 0.C93 
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2S. ASSlCV^iHITS CO:)rRtQL^:En 10 AI^PR£CI^MI0l|/UNnER3TANT)II« 


0 


0 


10 


48 


43 


2 


4.32 0.099 
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OVERALL 
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30- P3W DDES Ti;rS COURSE COMPARE ivITH aillERS AT U.S.C.? 


0 


0 


10 


36 


55 
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4.44 n.lC2 


'33 


***************** 
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31. X'E.> ':\^1J.5 !WT<CC10i^ CCKrARE >::Ti1 OH^ER-) A? U.S.C.? 
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29 


67 
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4.61 O.C'j9 
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